Address the review comments on #401 — the Go binding logs now follow the live
driver logger, but the Rust subscriber is process-wide/first-call-wins and
stderr-only, so several "one unified knob" claims were stronger than the
implementation. Fixes, all in the kernel backend (databricks_kernel tag; the
default pure-Go build is unaffected):

  - doc.go: Rust logs are stderr-only and NOT routed through logger.SetLogOutput;
    Rust verbosity is fixed at the first kernel session (set the level before the
    first connection); "each line carries connId/corrId/queryId" softened to
    request-scoped lines where a ctx is in scope (bindParams / operation teardown
    stay uncorrelated).
  - cgo.go: kernelLogLevel maps FatalLevel/PanicLevel -> OFF (not ERROR), so the
    Rust subscriber is never louder than the driver the user configured — at
    driver=fatal the Go side suppresses even Error() lines. Tightened the
    initKernelLogging comment to state the first-session level sampling and the
    stderr-only sink, and dropped the inaccurate "only installed when the level is
    at or below the threshold" sentence.
  - kernel_test.go: TestKernelLogLevel updated for fatal/panic->OFF; new
    TestKernelLogCtxEmitsCorrelation captures logger output at debug level and
    asserts klogCtx emits the message + connId/corrId/queryId, then asserts
    silence at warn — the positive-behavior half the alloc test can't see.

Verified: CGO_ENABLED=0 build+vet+test green; CGO_ENABLED=1 -tags databricks_kernel
build+vet green, kernel package tests pass (incl. all three log tests).

Co-authored-by: Isaac
Signed-off-by: Mani Kaustubh Mathur <mani.mathur@databricks.com>

Follow-up to the #401 review (finding 5, thread on kernel_test.go): the level-
resolution logic initKernelLogging feeds to kernel_init_logging was documented but
not tested. Extract the pure decision so it can be pinned by CI without cgo.

  - New untagged logging_level.go holds kernelLogLevel (moved out of cgo.go) and a
    new resolveKernelLogArg() (level string, useNULL bool): the NULL-on-
    DBSQL_KERNEL_DEBUG override vs the mapped driver level. Mirrors the existing
    errors_classify.go pattern (pure logic, no build tag) so its tests run under
    CGO_ENABLED=0. cgo.go's initKernelLogging is now a thin caller of it.
  - New untagged logging_level_test.go: TestKernelLogLevel (moved) + new
    TestResolveKernelLogArg, which pins the two invariants the PR justifies in prose
    — DBSQL_KERNEL_DEBUG (non-empty) yields useNULL=true so the kernel honors
    RUST_LOG, and otherwise the driver level is mapped in (incl. fatal→OFF). Empty
    value is treated as unset.
  - Drops the now-unused os import from cgo.go and zerolog from kernel_test.go.

Not covered (documented, cost/value flips): that the string is actually handed to
C.kernel_init_logging and the sync.Once first-call-wins — asserting those needs a
function-pointer seam over the cgo call plus resetting a package global, to test one
assignment + stdlib sync.Once.

Verified: CGO_ENABLED=0 build+vet+test green (both new tests run here); CGO_ENABLED=1
-tags databricks_kernel build+vet green, all four log tests pass.

Co-authored-by: Isaac
Signed-off-by: Mani Kaustubh Mathur <mani.mathur@databricks.com>

Address the review comments on #401 — the Go binding logs now follow the live
driver logger, but the Rust subscriber is process-wide/first-call-wins and
stderr-only, so several "one unified knob" claims were stronger than the
implementation. Fixes, all in the kernel backend (databricks_kernel tag; the
default pure-Go build is unaffected):

  - doc.go: Rust logs are stderr-only and NOT routed through logger.SetLogOutput;
    Rust verbosity is fixed at the first kernel session (set the level before the
    first connection); "each line carries connId/corrId/queryId" softened to
    request-scoped lines where a ctx is in scope (bindParams / operation teardown
    stay uncorrelated).
  - cgo.go: kernelLogLevel maps FatalLevel/PanicLevel -> OFF (not ERROR), so the
    Rust subscriber is never louder than the driver the user configured — at
    driver=fatal the Go side suppresses even Error() lines. Tightened the
    initKernelLogging comment to state the first-session level sampling and the
    stderr-only sink, and dropped the inaccurate "only installed when the level is
    at or below the threshold" sentence.
  - kernel_test.go: TestKernelLogLevel updated for fatal/panic->OFF; new
    TestKernelLogCtxEmitsCorrelation captures logger output at debug level and
    asserts klogCtx emits the message + connId/corrId/queryId, then asserts
    silence at warn — the positive-behavior half the alloc test can't see.

Verified: CGO_ENABLED=0 build+vet+test green; CGO_ENABLED=1 -tags databricks_kernel
build+vet green, kernel package tests pass (incl. all three log tests).

Co-authored-by: Isaac
Signed-off-by: Mani Kaustubh Mathur <mani.mathur@databricks.com>

Follow-up to the #401 review (finding 5, thread on kernel_test.go): the level-
resolution logic initKernelLogging feeds to kernel_init_logging was documented but
not tested. Extract the pure decision so it can be pinned by CI without cgo.

  - New untagged logging_level.go holds kernelLogLevel (moved out of cgo.go) and a
    new resolveKernelLogArg() (level string, useNULL bool): the NULL-on-
    DBSQL_KERNEL_DEBUG override vs the mapped driver level. Mirrors the existing
    errors_classify.go pattern (pure logic, no build tag) so its tests run under
    CGO_ENABLED=0. cgo.go's initKernelLogging is now a thin caller of it.
  - New untagged logging_level_test.go: TestKernelLogLevel (moved) + new
    TestResolveKernelLogArg, which pins the two invariants the PR justifies in prose
    — DBSQL_KERNEL_DEBUG (non-empty) yields useNULL=true so the kernel honors
    RUST_LOG, and otherwise the driver level is mapped in (incl. fatal→OFF). Empty
    value is treated as unset.
  - Drops the now-unused os import from cgo.go and zerolog from kernel_test.go.

Not covered (documented, cost/value flips): that the string is actually handed to
C.kernel_init_logging and the sync.Once first-call-wins — asserting those needs a
function-pointer seam over the cgo call plus resetting a package global, to test one
assignment + stdlib sync.Once.

Verified: CGO_ENABLED=0 build+vet+test green (both new tests run here); CGO_ENABLED=1
-tags databricks_kernel build+vet green, all four log tests pass.

Co-authored-by: Isaac
Signed-off-by: Mani Kaustubh Mathur <mani.mathur@databricks.com>

Address the review comments on #401 — the Go binding logs now follow the live
driver logger, but the Rust subscriber is process-wide/first-call-wins and
stderr-only, so several "one unified knob" claims were stronger than the
implementation. Fixes, all in the kernel backend (databricks_kernel tag; the
default pure-Go build is unaffected):

  - doc.go: Rust logs are stderr-only and NOT routed through logger.SetLogOutput;
    Rust verbosity is fixed at the first kernel session (set the level before the
    first connection); "each line carries connId/corrId/queryId" softened to
    request-scoped lines where a ctx is in scope (bindParams / operation teardown
    stay uncorrelated).
  - cgo.go: kernelLogLevel maps FatalLevel/PanicLevel -> OFF (not ERROR), so the
    Rust subscriber is never louder than the driver the user configured — at
    driver=fatal the Go side suppresses even Error() lines. Tightened the
    initKernelLogging comment to state the first-session level sampling and the
    stderr-only sink, and dropped the inaccurate "only installed when the level is
    at or below the threshold" sentence.
  - kernel_test.go: TestKernelLogLevel updated for fatal/panic->OFF; new
    TestKernelLogCtxEmitsCorrelation captures logger output at debug level and
    asserts klogCtx emits the message + connId/corrId/queryId, then asserts
    silence at warn — the positive-behavior half the alloc test can't see.

Verified: CGO_ENABLED=0 build+vet+test green; CGO_ENABLED=1 -tags databricks_kernel
build+vet green, kernel package tests pass (incl. all three log tests).

Co-authored-by: Isaac
Signed-off-by: Mani Kaustubh Mathur <mani.mathur@databricks.com>

Follow-up to the #401 review (finding 5, thread on kernel_test.go): the level-
resolution logic initKernelLogging feeds to kernel_init_logging was documented but
not tested. Extract the pure decision so it can be pinned by CI without cgo.

  - New untagged logging_level.go holds kernelLogLevel (moved out of cgo.go) and a
    new resolveKernelLogArg() (level string, useNULL bool): the NULL-on-
    DBSQL_KERNEL_DEBUG override vs the mapped driver level. Mirrors the existing
    errors_classify.go pattern (pure logic, no build tag) so its tests run under
    CGO_ENABLED=0. cgo.go's initKernelLogging is now a thin caller of it.
  - New untagged logging_level_test.go: TestKernelLogLevel (moved) + new
    TestResolveKernelLogArg, which pins the two invariants the PR justifies in prose
    — DBSQL_KERNEL_DEBUG (non-empty) yields useNULL=true so the kernel honors
    RUST_LOG, and otherwise the driver level is mapped in (incl. fatal→OFF). Empty
    value is treated as unset.
  - Drops the now-unused os import from cgo.go and zerolog from kernel_test.go.

Not covered (documented, cost/value flips): that the string is actually handed to
C.kernel_init_logging and the sync.Once first-call-wins — asserting those needs a
function-pointer seam over the cgo call plus resetting a package global, to test one
assignment + stdlib sync.Once.

Verified: CGO_ENABLED=0 build+vet+test green (both new tests run here); CGO_ENABLED=1
-tags databricks_kernel build+vet green, all four log tests pass.

Co-authored-by: Isaac
Signed-off-by: Mani Kaustubh Mathur <mani.mathur@databricks.com>

Address the review comments on #401 — the Go binding logs now follow the live
driver logger, but the Rust subscriber is process-wide/first-call-wins and
stderr-only, so several "one unified knob" claims were stronger than the
implementation. Fixes, all in the kernel backend (databricks_kernel tag; the
default pure-Go build is unaffected):

  - doc.go: Rust logs are stderr-only and NOT routed through logger.SetLogOutput;
    Rust verbosity is fixed at the first kernel session (set the level before the
    first connection); "each line carries connId/corrId/queryId" softened to
    request-scoped lines where a ctx is in scope (bindParams / operation teardown
    stay uncorrelated).
  - cgo.go: kernelLogLevel maps FatalLevel/PanicLevel -> OFF (not ERROR), so the
    Rust subscriber is never louder than the driver the user configured — at
    driver=fatal the Go side suppresses even Error() lines. Tightened the
    initKernelLogging comment to state the first-session level sampling and the
    stderr-only sink, and dropped the inaccurate "only installed when the level is
    at or below the threshold" sentence.
  - kernel_test.go: TestKernelLogLevel updated for fatal/panic->OFF; new
    TestKernelLogCtxEmitsCorrelation captures logger output at debug level and
    asserts klogCtx emits the message + connId/corrId/queryId, then asserts
    silence at warn — the positive-behavior half the alloc test can't see.

Verified: CGO_ENABLED=0 build+vet+test green; CGO_ENABLED=1 -tags databricks_kernel
build+vet green, kernel package tests pass (incl. all three log tests).

Co-authored-by: Isaac
Signed-off-by: Mani Kaustubh Mathur <mani.mathur@databricks.com>

Follow-up to the #401 review (finding 5, thread on kernel_test.go): the level-
resolution logic initKernelLogging feeds to kernel_init_logging was documented but
not tested. Extract the pure decision so it can be pinned by CI without cgo.

  - New untagged logging_level.go holds kernelLogLevel (moved out of cgo.go) and a
    new resolveKernelLogArg() (level string, useNULL bool): the NULL-on-
    DBSQL_KERNEL_DEBUG override vs the mapped driver level. Mirrors the existing
    errors_classify.go pattern (pure logic, no build tag) so its tests run under
    CGO_ENABLED=0. cgo.go's initKernelLogging is now a thin caller of it.
  - New untagged logging_level_test.go: TestKernelLogLevel (moved) + new
    TestResolveKernelLogArg, which pins the two invariants the PR justifies in prose
    — DBSQL_KERNEL_DEBUG (non-empty) yields useNULL=true so the kernel honors
    RUST_LOG, and otherwise the driver level is mapped in (incl. fatal→OFF). Empty
    value is treated as unset.
  - Drops the now-unused os import from cgo.go and zerolog from kernel_test.go.

Not covered (documented, cost/value flips): that the string is actually handed to
C.kernel_init_logging and the sync.Once first-call-wins — asserting those needs a
function-pointer seam over the cgo call plus resetting a package global, to test one
assignment + stdlib sync.Once.

Verified: CGO_ENABLED=0 build+vet+test green (both new tests run here); CGO_ENABLED=1
-tags databricks_kernel build+vet green, all four log tests pass.

Co-authored-by: Isaac
Signed-off-by: Mani Kaustubh Mathur <mani.mathur@databricks.com>

…tly workflows for e2e tests (#399)

## What

Extends the SEA-via-kernel backend for the Go driver with the full DAIS
gap-closure
+ PuPr feature set: authentication, session setup, type rendering, query
execution,
telemetry, richer TLS, kernel-log routing, and CI. 

This PR consolidates what was a 5-PR stack (#400 richer-TLS, #401
log-unify,
#402 nightly-e2e, #403 c2-dispatch) into a single reviewable PR. The
stack was linear,
so the branches were fast-forwarded into this one — every change is
preserved as a
**distinct, logically-scoped commit** (23 commits), so it reads
commit-by-commit rather
than as one blob. #400–#403 now show as merged (their commits live
here).

## Features

**Authentication**
- **OAuth M2M / U2M** — the kernel drives its own auth flow from
`cfg.Authenticator`
(mirroring pyo3/napi). Adds a `kernel.Auth` descriptor +
`resolveKernelAuth`, wired
through the `set_auth_pat` / `_m2m` / `_u2m` C-ABI setters.
`cfg.Authenticator` is the
  single source of truth (last-writer-wins, matching Thrift).
- **Unsupported auth is sentinel-wrapped** — token-provider / external /
federated
authenticators reject with `%w`-wrapped `ErrNotSupportedByKernel`, the
same
programmatic-fallback contract every other unsupported kernel option
follows.

**Session setup**
- **Initial namespace** — post-connect `USE CATALOG` / `USE SCHEMA` with
identifier
  quoting (the kernel C ABI has no namespace setter).
- **Metric-view metadata** — `config.EffectiveSessionParams()` folds the
server conf in
backend-neutrally, so the kernel path sends the identical conf Thrift
does.

**Type rendering**
- **INTERVAL** day-time & year-month rendering in `internal/arrowscan`
(kernel returns
native arrow values; formatted Go-side to the Thrift path's string
form). Handles the
  `math.MinInt64` / `math.MinInt32` negation bound without overflow.
- Parity coverage for **TIMESTAMP vs TIMESTAMP_NTZ** and **VARIANT /
GEOMETRY**
  (live-probed on both backends).

**Query execution & telemetry**
- **Bound query parameters** via the kernel raw-param C ABI
(`kernel_statement_bind_parameter`). The positional/named +
SQL-NULL/empty-string
decision is a pure `paramBindArg` seam, unit-tested under
`CGO_ENABLED=0`.
- **Kernel errors surfaced as `DBExecutionError`** carrying sqlstate +
server query id,
so `errors.As` → `SqlState()` / `QueryId()` works on the kernel path.
The execute
path deliberately reports **non-retryable** (a sent statement may have
committed — no
  double-write), matching Thrift.
- **Server query id** via a real `StatementID()` accessor
(`kernel_executed_statement_query_id`), threaded into EXECUTE_STATEMENT
telemetry, plus
  CLOSE_STATEMENT telemetry on the result-read path.
- **Cancelled execute** evicts a session-fatal conn and preserves the
kernel error
metadata (multi-`%w`: `errors.Is(DeadlineExceeded)` AND
`errors.As(*KernelError)` both
  hold).

**Richer TLS (Go options)**
- CA-bundle + independent hostname-skip setters mapped to the kernel's
own TLS knobs
(defensive PEM copy; the wholesale custom `WithTransport` stays
rejected).

**Kernel logging**
- Kernel log output routed through the driver log level with
connId/corrId/queryId
correlation; allocation-free above Debug (guards the Arrow-batch hot
path).
- **scope note (please read before flagging):**
- This unifies the *control knob*, not the *sink*, and that boundary is
intentional.
`DATABRICKS_LOG_LEVEL`/`SetLogLevel` now drives both the Go binding
lines and the
kernel's Rust lines from one level. But they stay two physical streams:
Go lines go
through `logger.Logger` (and honor `logger.SetLogOutput`), while the
kernel's Rust
lines go straight to stderr — the kernel C ABI exposes **no
sink/callback hook**, so
there is currently no way to route them through the Go logger.
Consequences a
reviewer will notice, all expected:
- A custom `SetLogOutput`/file captures only the Go lines; Rust lines
stay on stderr.
- With the default sink both land on stderr but only *race* there — not
a single
  ordered, merged stream.
- The Rust subscriber's level is sampled once, at the first kernel
connect.

Fully merging both into one ordered sink requires a kernel log callback
and is
tracked separately (<TICKET>); it's out of scope here because it's a
kernel-side
ABI change, not a driver one.

**CI**
- A scheduled + dispatchable **nightly E2E** workflow that runs the
credential-gated
e2e + Thrift-vs-kernel parity suites against a real warehouse (the
ordinary CI injects
no warehouse secrets, so those tests otherwise skip). SEA-via-kernel
integration leg
  gated behind a kernel label.

## Notable behavior

- Bound query parameters are now supported (previously rejected at
connect).
- **Staging operations remain rejected** — the kernel path can't perform
the local file
  transfer and the C ABI surfaces no staging signal.

## Testing

- **Default pure-Go** (`CGO_ENABLED=0`): `go build` / `go vet` / `go
test ./...` — 24
  packages ok.
- **Tagged kernel path** (`CGO_ENABLED=1 -tags databricks_kernel`,
linked against a
  locally-built kernel `.a` at `KERNEL_REV`): 24 packages ok.
- **Live e2e + parity** against a real warehouse: all pass — data types
(incl.
variant/geometry/decimal), NTZ, params-vs-Thrift (10/10), CloudFetch,
cancellation,
connection pool (40/40), StatementID, initial namespace, and **M2M**
(OAuth
  client-credentials).
- `gofmt` clean; Isaac Review clean (0 final comments).

## Merge gate

- **Re-pin `KERNEL_REV` at merge time.** It currently points at the
kernel dependency's
PR-head SHA (GC-able once that kernel change merges); bump it to the
resulting kernel
`main` SHA, re-sync the cgo drift assertions in `cgo.go` if any
signatures changed, and
  run `make test-kernel` against the new rev.

This pull request and its description were written by Isaac.

---------

Signed-off-by: Mani Kaustubh Mathur <mani.mathur@databricks.com>